Appendix for The Queue Method: Handling Delay, Heuristics, Prior Data, and Evaluation in Bandits

Proof. Let T ′ be the number of times we have updated BASE. Note that since the samples fed to BASE were drawn iid according to the arm distributions and given on the arms it requested, the regret on these samples is RT ′ . Clearly T ′ ≤ T since for every sample we give BASE, we must have correspondingly taken a step in the true environment1. So the expected regret of only those steps in which we update BASE can be upper bounded by